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(57) Abstract 

A method and apparatus for magnifying a portion P1-P6 of a digital image on a display screen in either of two ways. The first 
method (Figs. 5 and 10) includes a two pass scheme, where each of the passes represents an interpolation in x and y direction respectively, 
cubic interpolation in each direction is approximated using a one dimensional convolution filter (56) followed by linear interpolation (58). 
The second method (figs. 8 and 11) uses a two dimensional convolution filter (90) first, followed by bilinear interpolation (92). All of the 
procedures that are used are accelerated using a hardware package which facilitates exceptionally fast execution. 
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METHOD AND APPARATUS FOR ZOOMING DIGITAL IMAGES 



CROSS-REFERENCE TO RELATED APPLICATIONS 
Not applicable. 



STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR 
DEVELOPMENT 



Not applicable. 



BACKGROUND OF THE INVENTION 
5 The present invention relates to imaging methods and apparatus and 

more specifically to a method and apparatus which facilitates magnification or 
zooming of a portion of an image on a display. 

Digital imaging is extremely important in many different applications. 
For example, digital imaging has proven invaluable in the medical imaging art 

10 where vast amounts of data are collected and used to generate images for 
observation on a video monitor or display. Although not so limited, in the 
interest of simplifying this explanation the present invention will be described 
in the context of medical imaging. A typical display includes a two- 
dimensional raster of pixels. For the purpose of this explanation, although 

15 raster pixels may be arranged in any of several different configurations, it will 
be assumed that pixels are arranged in distinct rows and columns. 

To generate images on the display a processor collects all image data 
and generates intensity signals or pixel values for each display pixel. The 
pixel values are provided to a video driver which excites each pixel separately 

20 in.accordance with an associated pixel value. From a distance the contrast 
between display pixel intensities is seen as an image. 

Magnification of digital images in real time is needed in many different 
applications. For example, it may be advantageous to magnify a section of a 
medical image which includes a tumor. To this end, software has been 
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developed which allows a user to select an image section for magnification 
and then magnifies the selected section. 

One problem with image magnification has been selecting pixel 
intensities in a magnified image which reflect the initial image and result in a _ 
5 clean magnified image. For example, assume an intensity range between 0 
and 100 where first and second adjacent pixel values correspond to intensities 
55 and 92. Also assume that upon magnification, the area corresponding to 
the first and second pixels increases and covers 32 pixels (e.g. 16 pixels 
each). In this case, if the 16 magnified pixels corresponding to the first pixel 

10 and the 16 magnified pixels corresponding to the second pixel are provided 
with intensities 55 and 92, respectively, and all other magnified pixels are 
similarly magnified, the result is a highly granulated image which, in many 
cases, is not useful for the intended purpose of closer examination. 

Instead of exciting pixels in the magnified image as indicated above 

15 (i.e. with either intensities 55 or 92), other solutions have been adopted by the 
industry with varying success and at varying costs. The most common 
methods for calculating new pixel intensities are nearest neighbor, bilinear 
and bicubic interpolation methods. 

According to the nearest neighbor method, when points corresponding 

20 to first and second adjacent pixels on an initial image are separated by 

magnification so that the initial image points correspond to third and fourth 
pixels which are separated by a plurality of other pixels, the intensity of each 
of the other pixels (e.g. pixels between the third and fourth) are set equal to 
the intensity of the closest of the third or fourth pixel. This solution has the 

25 advantage of being computationally simple and therefore can be implemented 
easily using existing imaging hardware. Unfortunately, nearest neighbor 
methods only increase magnified image quality slightly and therefore are 
unacceptable for many applications. 

Bilinear methods generally linearly fill in pixel intensities. For instance, 

30 in the example above where points corresponding to first and second adjacent 
pixels on an initial image are separated by magnification so that the initial 
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image points correspond to third and fourth pixels which are separated by 
three other pixels and the first and second pixel intensities were 55 and 92, 
respectively, the other pixel intensities are linearly determined and are 
approximately 64, 73 and 83. High speed bilinear image magnification is now_ 
commonly available in accelerated graphics hardware. Unfortunately, while 
this solution generates a better image than the nearest neighbor methods, this 
solution requires much more processor time to perform necessary 
computations and still does not provide an extremely accurate magnification. 

Bicubic interpolation methods generally take into account the intensities 
of more than just first and second adjacent pixels when determining the 
intensities of pixels in a magnified image which are between image points 
which correspond to the initial first and second pixels. In effect, these 
interpolation methods mathematically identify pixel intensities on one or more 
curves wherein the curves correspond to proximate initial pixel intensities of 
several pixels about an area. 

These interpolation methods are extremely accurate and generate 
diagnostic quality magnified images. Unfortunately, these methods require 
massive amounts of processor time and therefore, in many cases, cannot be 
performed in real time because of processor limitation. For this reason high 
quality bicubic image magnification is not generally available. 

One way to speed up calculations is to provide special hardware which 
is specifically designed to perform specific calculations. For example, many 
image processing systems include special hardware to perform either one or 
two dimensional high speed convolution filtering and linear or bilinear 
interpolation processes required in many imaging application. Unfortunately, 
hardware solutions have not yet been provided to facilitate bicubic 
interpolation. 

Therefore, it would be advantageous to have a method and an 
apparatus which can be used with existing hardware to facilitate image 
magnification wherein resulting magnified images are of a quality which is 
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essentially identical to the quality achievable using bicubic interpolation and 
wherein the method facilitates real time magnification. 

BRIEF SUMMARY OF THE INVENTION 

It has been recognized that specific hardware which already exists in 
many imaging work stations can be used to imitate bicubic interpolation 
thereby providing extremely accurate image magnification. Importantly, while 
the functions and calculations performed by the hardware are extremely 
computationally intensive, the special hardware can perform the functions and 
calculations extremely quickly thereby giving the appearance of real time 
zooming or magnification. 

Generally, the hardware includes a convolver, an interpolator, and a 
processor for identifying convolver coefficients. After the coefficients are 
identified, the convolver convolves pixel intensities for each pixel in the 
original image region to be magnified thereby generating at least two 
intermediate values. The interpolator interpolates the intermediate values to 
generate output pixel intensities. 

Specifically, according to a one dimensional method of zooming, the 
invention includes a method and apparatus to be used with a digital imaging 
system to facilitate magnification of a region of interest on an initial digital 
image wherein the region of interest includes a plurality of pixels of interest, 
adjacent halves of adjacent pixels of interest forming interpixel intervals of 
interest. Each pixel of interest is characterized by a pixel intensity. The 
system includes convolution filter and interpolation hardware. The method 
comprises the steps of, for each interpixel interval of interest, identifying 
adjacent pixels, for each adjacent pixel, identifying a convolution window and 
for each window, determining a convolution filter coefficient for each pixel in 
the window, convoluting the pixel intensities of the pixels in each window 
using the coefficients to generate an intermediate value, and interpolating the 
intermediate values to generate interpolated pixel intensities, the interpolated 
pixel intensities together forming an interpolated image. 
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Preferably, according to the one-dimensional method, the initial pixel 
array is arranged in y columns and x rows and the step of selecting includes 
the step of, for each interpixel interval of interest between pixels in adjacent 
columns, selecting first and second row convolution windows consisting of _ 
5 pixels which are in the same row as the interval of interest and wherein, the 
method further includes the steps of, for each interpixel interval between 
pixels in adjacent rows on the interpolated image, selecting first and second 
separate column convolution windows on the interpolated image, each column 
window including adjacent image pixels within the same column as, and 

10 proximate the interpixel interval of interest, for each window, determining a 
convolution filter coefficient for each pixel in the window and convoluting the 
pixel intensities in each window using the coefficients to generate an 
intermediate value, and interpolating the intermediate values to generate a 
final pixel intensity, the final pixel intensities together forming the final image. 

15 The invention also includes a two-dimensional method and apparatus for 
zooming which is similar to the one-dimensional method and apparatus. 

In the description, reference is made to the accompanying drawings 
which form a part hereof, and in which there is shown a preferred embodiment 
of the invention. Such embodiment does not necessarily represent the full 

20 scope of the invention and reference is made therefore, to the claims herein 
for interpreting the scope of the invention. 

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS 
Fig. 1 is a schematic view illustrating one dimensional convolution 

interpolation methods according to the present invention; 

Fig. 2 is a is a schematic illustrating a convolution window in 
25 accordance with a second embodiment of the present invention; 

Fig. 3 is a schematic view illustrating a relationship between 

intermediate intensity values and all pixel intensities according to the second 

embodiment to the invention; 
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Fig. 4 is a perspective view of a computer system used to facilitate the 
inventive methods; 

Fig. 5 is a schematic diagram of hardware used in both the first and 
second embodiments to the present invention; 

Fig. 6 is a schematic illustrating hardware according to a one 
dimensional convolution embodiment of the present invention; 

Fig. 7 is a schematic diagram illustrating a region of interest and 
expanded region of interest according to the present invention; 

Fig. 8 is similar to Fig. 6, albeit illustrating hardware required for a two- 
dimensional convolution system according to the present invention; 

Fig. 9 is a flow chart illustrating operation of the hardware of Fig. 5; 

Fig. 10 is a flow chart illustrating operation of the hardware of Fig. 6; 

and 

Fig. 1 1 is a flow chart illustrating operation of the hardware of Fig. 8. 

DETAILED DESCRIPTION OF THE INVENTION 
A. THEORY 

Generally, it has been recognized that bicubic interpolation can be 
closely estimated or mimicked by performing a two-dimensional convolution of 
initial pixel intensities to generate intermediate values and then linearly 
interpolate between intermediate values to generate specific final image pixel 
intensities. While such convolution and interpolation procedures are 
computationally intensive, many imaging systems now include hardware 
which can perform required computations extremely quickly such that 
computations appear to be performed in real time. Specifically, the hardware 
includes convolution filters and linear or bilinear interpolators which can 
quickly provide the functions required to estimate pixel intensities which would 
be generated via software based bicubic interpolation. 

Some systems only include hardware which can be used to perform 
either one or two-dimensional convolution and either linear or bilinear 
interpolation. The present invention includes a first method referred to herein 
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as a one-dimensional convolution (ODC) method which can be used by 
.hardware which is limited to one-dimensional convolution and linear 
interpolation to estimate output pixel intensities. In addition, the present 
invention also includes a second method referred to herein as a two- 
5 dimensional convolution (TDC) method which can be used by hardware 
which can perform two-dimensional convolution and bilinear interpolation to 
estimate output pixel intensities. Each of the one and two-dimensional 
convolution methods is described separately below. 

In each of the methods below it will be assumed that display pixels are 

10 arranged in x horizontal rows and y vertical columns. In addition, it will be 

assumed that a command received from a user indicates that a specific image 
region should be magnified by a factor n where n is 5 (i.e. magnification is by 
a factor of 5). Thus, generally, the area of a first image which corresponds to 
a single pixel will be magnified such that the same area in the final image 

1 5 corresponds to a block of pixels having 5x5 dimensions. The region to be 
magnified will be referred to herein as a region of interest and each pixel 
within the region of interest will be referred to as a pixel of interest. 

1 . ONE-DIMENSIONAL CONVOLUTION (ODC) METHOD 
20 According to the ODC method, when a command is received to 

magnify region of interest by a factor of n, prior to convolution and 
interpolation an expanded region of interest which includes the region of 
interest and an additional number of rows above and below the region of 
interest is selected. The additional rows in the expanded must be processed 
25 in the x dimension so that data is generated for processing in the y dimension. 
This requirement will become more clear below. 

After the expanded region has been selected, convolution and 
interpolation hardware is used to modify pixel data first in the x (i.e. row) 
dimension to generate an interpolated image. Thereafter, the interpolated 
30 image data is fed again to the convolution and interpolation hardware which 
modifies the pixel data in the y (i.e. column) dimension thereby generating a 
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final or output image. Each pass through the hardware includes a two step 
process corresponding to each pixel processed during the pass. In the first 
step, for each pixel in the image being processed the convolution hardware 
selects a separate convolution window on the image being processed (i.e. on - 
5 the initial image during the first pass and on the interpolated image during the 
second pass). During the first pass each window (i.e. a row window) is in the 
same row as a corresponding pixel of interest and during each second pass 
each window (i.e. a column window) is in the same column as a 
corresponding final pixel. In addition, each window includes pixels adjacent 

10 the pixel of interest. The number of pixels in each convolution window 

depends on a filter size m where m is typically a number of pixels (e.g. 5, 7, 
etc.). Hereinafter m will be assumed to be 5 unless indicated otherwise. 

To better understand convolution windows and the inventive ODC 
method refer to Fig. 1 wherein six initial pixels P1 through P6 are illustrated. 

15 The pixels P1 through P6 are to be magnified by a factor of 5. For the 
purpose of this explanation, when referring to the ODC method, the term 
"interpixel interval" will be used to refer to adjacent halves of adjacent pixels. 
For example, in Fig. 1 , an interpixel interval corresponding to pixels P3 and P4 
is identified in phantom as P34. Similarly, there is an interpixel interval 

20 between each other two adjacent pixels. To magnify pixels P1 through P6, 
according to the ODC method each interpixel interval is magnified by a factor 
of 5. To this end, two convolution windows including m pixels are selected 
which correspond to each interval. For example, where a filter size m is 5, for 
interval P34, the windows are first window 10a including pixels P1 through P5 

25 and second window 10b including pixels P2 through P6. The intensities of 
pixels P1 through P6 are identified as I, through 1^ 

When the 5 times magnification request is received, in effect, the area 
in the initial image corresponding to each interpixel interval is first expanded in 
the x dimension to cover five pixels. For example, in Fig. 1 , the area 

30 corresponding to interpixel interval P34 is expanded into interpolated pixels 
PV through P5' having intensities 01 through OS forming an output pixel 
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block 12. Similarly, the interval areas between other initial adjacent pixel pairs 
P1 and P2, P2 and P3, P4 and P5 and P5 and P6 are each expanded into five 
distinct interpolated pixel intensities. Thus, pixels P1 through P6 are first 
expanded in the x dimension to form 25 separate interpolated pixels (i.e. 5 
output pixels corresponding to each interpixel interval), each output pixel 
characterized by a separate intensity. Thereafter, the same procedure is 
repeated in the y dimension to provide five separate output pixel intensities for 
the area between any two adjacent interpolated pixels. 

Referring still to Fig. 1 , to expand interval P34 in the x dimension, 
convolution windows 10a and 10b are first selected about adjacent pixels P3 
and P4. After convolution windows 10a and 10b have been selected the 
convolution hardware convolutes each of the windows 10a and 10b 
separately, thereby generating two intermediate values b 1 and b 2 , 
respectively. The convolution equations for intermediate values b 1 and b 2 are 
expressed as: 



and 



(1) 



m 

b 2 = 



(2) 
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where a t are coefficients of the convolution filter which depend on the 
magnification factor n, where, in this example, n is 5. 

After each of intermediate values and b 2 have been identified, the 
intermediate values are provided to the linear interpolation hardware which 
interpolates intensities ^ and b 2 across n output pixels. To this end, the 
interpolation hardware performs the following equations to determine 
intensities O k of final pixels P1\ P2\ P3\ P4' and P5' where k is 1 through n, 
in the present example 1 through 5. Where n is odd: 



0l = (2^±l) 4l + (izi) 4! (3) 
n n 



and, where n is even: 



0 + 

n n 



Substituting values b, and b 2 from Equations 1 and 2 into Equations 3 
and 4 and simplifying yield, where n is odd: 



and where n is even: 



Equations 5 and 6 can be expressed as: 
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^II-W., (7) 

where the values are a function of the coefficients with corresponding I 

values from either Equation 3 or Equation 4, depending on if n is odd or even. 
For example, if n is odd and i 2 is 1 , A ih is from Equation 3 or: 

5 / i. I= ___ (8) 

The only unknowns in Equation 7 are convolution filter coefficients a u . 

Therefore, if coefficients a u can be identified for a specific magnification factor 

n, Equations 1 , 2 and 3 in the case of an odd factor n or, in the case of an 
even factor n, Equations 1 , 2 and 4, can be performed to determine intensities 
10 01-05 of pixels P1'-P5\ 

To identify the convolution filter coefficients a h , a set of equations 

similar to Equation 7 is first derived in accordance with conventional bicubic 
interpolation methods. Thereafter common coefficients of Equation 7 and the 
cubic Equations are set equal and solved for the convolution filter coefficients. 
15 To this end, as well known, where there are four equidistant points x„ x 2 , x 3 
and x 4 on an image and a function f defines intensity values l 1t l 2 , l 3 and l 4 of 
the four points, respectively, the intensity value of the function at some point 
x between points x 2 and x 3 can be determined as follows. First, assume dx is 
the normalized distance defined by: 

20 dx = ^±- (9) 

with distance dx so defined, the following cubic coefficients c, through c 4 can 
be calculated: 
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c x =-)-dx + Udx) 2 ~Udxy (10) 
5 1 0 

c 2 = l-^dx-(dx) 2 +±(dx) } (11) 

c 3 =dx + ±(dx) 2 -±(dxy (12) 

c A = -±dx + ±(dxy (13) 

The intensity value O k of function f at point x, using cubic interpolation can 
therefore be expressed as : 



=/(*) = !>,/, (14) 

Comparing Equations 7 and 14, and equating coefficients with 
corresponding input pixel intensities /„ a set of up to four linear equations for 
the convolution filter coefficients a. can be expressed as: 



(15) 



All four cubic coefficients q can be determined by solving Equations 10 
through 13 and, therefore, the only unknown values in Equation 15 are the 
five coefficients a 1 through a 5 . 

To generate additional equations for determining coefficients through 
a 5l the procedure above is repeated for each of the output pixels. Thus, 
referring again to Fig. 1 , because n is 5 and there are therefore five output 
pixels P1' through P5* in block 12, four q values are generated for each 
separate pixel P1' through P5 1 to generate twenty q values and appropriate 
equations like Equation 15, a separate equation for each of the twenty q 
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values. Thus, twenty (i.e. 4n) cubic equations are derived with only 5 (i.e. m) 
unknowns (i.e. a 1 through a 5 ). 

There can be less than four equations if the pixels obtained by bilinear 
interpolation used by the convolution filter do not have any contribution from 
5 some input pixels /,. In this case, the number of equations is reduced by the 
number of such pixels. 

The system of 4n equations can be expressed in matrix form as: 

C = AA (16) 

where C is the vector of corresponding coefficients c, from right sides of 
10 equations 15, A is the matrix of coefficients \ of those linear equations, and A 
is a vector of m filter coefficients a,. For practical values of zoom factors n and 
filter sizes m the matrix of Equation 16 represents an over determined system 
of linear equations. In this case, the solution to matrix Equation 16 is 
determined by using the linear least squares method which takes the form: 

15 A=(A T A)' 1 A T C (17) 

The linear least squares method and in particular Equation 17 are well 
known in mathematics and therefore will not be explained here in detail. For a 
general explanation of the linear least squares method reference should be 
had to any standard algebra text which teaches matrix algebra. 
20 Upon determining the coefficients a, corresponding to the convolution 

filter that is to be applied, the whole procedure for zooming digital images is 
defined. 

2. TWO-DIMENSIONAL CONVOLUTION (TDC) METHOD 

According to the TDC method, when a command is received to magnify 
25 a region of interest by factor n, a two dimensional hardware convolution filter 
and a bilinear interpolator are used to convolve and interpolate initial pixel 

13 
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intensities simultaneously in both the x and y dimensions thereby generating a 
final output image. 

Thus, unlike the ODC method which first processes values in one 
dimension and then processes values in a second dimension to magnify, the 
TDC method processes in both dimensions simultaneously. Other than this 
distinction and more complex mathematics which are required for 
simultaneous two-dimensional processing, the TDC method is very similar to 
the ODC method. 

For the purpose of this explanation, when referring to the TDC method 
the term "interpixel interval" will be used to refer to adjacent quarters of four 
adjacent pixels. Referring to Fig. 2, the general pixel scheme for applying the 
TDC method to a region of interest is illustrated. The initial region of interest 
includes a pixel block of (m+1) (m+1) pixels. In Fig. 2, \ u through l (m * 1){m+1) 
denote source image pixel intensities corresponding to pixels of interest P„ 
through P (m «. 1)(m+1)I b„ through b 22 denote intermediate values obtained by 
applying a 2D convolution filter to intensities !„ through l (m+1)(m+1) , numeral 20 
outlines a convolution filter window including pixels P„ through P mm and 0„ 
through O nn represent intensities of output pixels P n through P M obtained by 
bilinear interpolation between intermediate pixels b 11t b 12 , b 21l and b 22 . 
Generally intensities l„ through l mm will be referred to as l ir 

Intermediate values b n through b 22 are obtained by applying the mxm 
convolution filter to the input pixels / tf according to the following expressions: 



m m 

ZZv* 



(18) 



m m 



'12 



(19) 



m m 



hi 



'21 



(20) 
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where a 9 are the coefficients of the two-dimensional convolution filter. 

After intermediate values b n through are determined bilinear 
interpolation is applied to generate output intensities O u through O nn . Fig. 3 
illustrates the relationship between intermediate values b 11t b 12 , b 21 , and b^ 
and the output pixels O u through O nn . Each of the output pixels O l} occupies a 
different location relative to the intermediate values b u through b 22 . All the 
other output pixels generated by the linear interpolation occupy the same 
relative locations to some intermediate pixels. The values of the output pixels 
0„ through O nn , with respect to the intermediate values are given by the 
following equations: 



_ («-i + l)(n-y + l) , (*-/ + l)C/-l) L 

0 iJ ~ "3 b U + ~2 °\2 + 

n n (22) 

(f - l)(n -y + l) L 0-1)0-1) L 
j b lx + - b : 



2 "21 ' 2 "22 

n n 



for 1,1=1,..., n (n odd), and 

- (« - i + Q.5)(n - j + 0.5) , (» - i + 0.5)(y - 05) , 

°# = "1 b n + "3 b n + 

n n (23) 

(,-Q5)(H-y + 0i) L , (i-05)(/-0.S) L 

^ *2I+ "2 6 22 

fori, j=1, n (n even). 

When the expressions for intermediate values b„ through b^from 
Equations 18 through 21 are substituted in Equation 22, the following 
expression is obtained: 
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°* ~ 2-2- <V« + 2 2-2- 'Vl'+I + 

(24) 

i 2-2- a « y *M/ + 5 — 2-2- a « 7 * 



(/-!)(*- y+l) ^_ , ,(i-ix/-i)^ 



which can be expresses as: 



m+l m+l m m 



°« = ISIZ^v,y.%^ (25) 

l ( =l l 2 =l i' 4 =l 



where are the constant coefficients from Equations 22. Similar 

expressions are obtainable using Equations 23. 

As well known in the art, using known expressions for bicubic 
interpolation alternate expressions for pixel values O kl can be obtained which 
can be collectively expressed as: 



0* = Z2>fA (26) 



where Cy are the coefficients derived from the expressions for bicubic 
interpolation in the case of magnification by n. Comparing Equations 25 and 
26 and equating coefficients with corresponding input pixel values /„, a set of 
up to sixteen linear equations for the filter coefficients a iiU are obtained which 

can be collectively expressed as: 

It^u (27) 
ij-H 4 -i 

There can be less than sixteen equations if the pixels obtained by bilinear 
interpolation used by the convolution filter do not have any contribution from 
some input pixels In that case, the number of equations is reduced by the 
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number of such pixels. Repeating this procedure for all of the output pixels Oy 
(i, j=1 , n) a system of (16n) linear equations for the convolution filter 
coefficients is obtained which can be expressed in matrix form as: 

C = AA (28) 

5 where C is a vector of the corresponding coefficients c l} from right side of 

Equations 26, A is the matrix of coefficients X ijkl of those linear equations, and 
A is the vector of m x m filter coefficients a„ through a mm . For practical values 
of zoom factors n and filter sizes m by m this represents an over determined 
system of linear equations. In this case, the coefficients a, can be determined 
10 using the linear least squares method indicated by Equation 17 above: 

Upon determining the coefficients a, of the convolution filter that is to 
be applied, the whole procedure for zooming digital images is defined. 

B. HARDWARE AND OPERATION 

Referring now to Fig. 4, a conventional workstation 30 is illustrated 

15 which includes a computer 32, a display 34 linked to the computer 32, and 
two interface devices, a keyboard 36 and a mouse 38. Keyboard 36 and 
mouse 38 allow a user to provide information to the computer including 
commands to control the computer 32. Information including images, is 
displayed on display 34 for observation. For the purpose of this explanation it 

20 is assumed that an image is displayed via display 34 and that a workstation 
user uses either board 36 or mouse 38 to select a region of interest on the 
initially displayed image to be magnified by a factor of 5 (i.e. n=5). 

In addition to other processing hardware, computer 32 must include 
hardware capable of either one-dimensional convolution filtering and linear 

25 interpolation or two-dimensional convolution filtering and bilinear interpolation. 
To this end, referring to Fig. 5, computer 32 includes an interpolation 
coefficient calculator 40, a cubic coefficient calculator 42 and a linear least 
square determiner 44. Referring also to Fig. 9, at block 160, calculator 40 
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receives magnification factor n and uses factor n to determine interpolation 
coefficients. To this end, on one hand, where a hardware based convolver is 
only capable of one dimensional convolution and a hardware based 
interpolator is only capable of linear interpolation, calculator 40 uses the 
5 coefficient expressions from Equations 5 and 6 to determine interpolation 
coefficients at block 162. For example, where n is 5 and m is 6 the four 
equations corresponding to Equation 15 above for the first output pixel (i.e. 
n=1) are: 

/l tt a t +/l t2 a 2 +/l t3 a3+/l, 4 a 4 +/l, 5 a 5 =c t (29) 
1 0 A 21 a t +/l 22 a 2 +/l 2 3a3+/L 24 a 4 +/l 25 a 5 =c 2 (30) 

^3f^f + ^32^2 +/ ^33^3 +/ ^34^4 +/ ^35^5 = ^3 (31 ) 

A 41 d 1 +A 42 32'* r J, 43 S2+A 44 3 4 +A 45 d 5 —C 1 (32) 

Looking at Equation 5 for I, where / is 1 corresponding to Equation 29, there is 
only a single coefficient expression for a 1t the expression being (n-k+1 )/n and 

1 5 therefore coefficient X^ is known to be (n-k+1 )/n and all other coefficients A. 12l 
^13. ^14 and X 15 are known to be zeros. Where / is 2 corresponding to Equation 
30, referring to Equation 5, coefficient X 2 i is (k-1)/n, coefficient A, 22 is (n-k+1 )/n 
and all other X coefficients in Equation 30 are zero. Where / is 3 
corresponding to Equation 31 , coefficient X 31 is zero, coefficient A, 32 is (k-1 )/n, 

20 coefficient X 33 is (n-k+1 )/n and all other coefficients X in Equation 31 are zeros. 
Where / is 4 corresponding to Equation 32 coefficients X 41 and X 42 are zeros, 
coefficient X 4Z is (k-1)/n, coefficient X u is (n-k+1 )/n and all other coefficients 
are zeros. The k value used in each of the coefficients corresponds to an 
output pixel and Equations like 29 through 32 are examined for each intended 

25 output pixel thereby generating 4n interpolation coefficients X which are 
provided to determiner 44. 
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On the other hand, where the hardware convolver is a two dimensional 
convolver and a bi-linear interpolator is provided, at block 162 calculator 40 
uses Equations 22 or 23, depending on if factor n is odd or even, to determine 
interpolation coefficients \ p which are provided to determiner 44. 
5 Referring still to Figs. 5 and 9, at block 164 cubic coefficient calculator 

42 also receives factor n and determines bicubic coefficients. Where the 
system convolver facilitates only one dimensional convolution calculator 42 
uses Equations 9 and 10 through 13 or other similar equations to determine 
coefficients c t through c 4 for each separate output pixel. Where the system 

10 can perform two dimensional convolution, calculator 42 generates sixteen 
coefficients q through c 16 for each output pixel using conventional bicubic 
equations which are well known in the art. Thus, 16n coefficients are 
generated and provided to determiner 44. 

In addition to receiving interpolation coefficients X and cubic 

15 coefficients c, determiner 44 also receives a value m indicating the filter 

dimensions. At blocks 166 and 168 determiner 44 forms the matrix equation 
indicated by Equation 17 and solves the equation to generate m or mxm 
coefficients a which are required in either Equations 1 and 2 or in Equations 
18, 19, 20 and 21 . Coefficients a, or a, y are provided as an output from 

20 determiner 44. 

The remaining hardware in the case of the ODC method is illustrated in 
Fig. 6 while the remaining hardware in the case of the TDC method is 
illustrated in Fig. 7. Referring to Fig. 6, to facilitate the ODC method computer 
32 ( see Fig. 5) further includes a 1D convolver 56, a linear interpolator 58 and 

25 a data buffer 60. 

Referring also to Fig. 10 at block 180, convolver 56 receives each of 
the initial image pixel intensity /, which corresponds to the expanded region of 
interest. Referring also to Fig. 7, a region of interest 80 (i.e. region of an 
image to be magnified) includes A rows and B columns. To convolve and 

30 interpolate convolution windows for each interval in the region of interest 80 in 
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the x dimension (i.e. horizontal), the windows include m+1 pixel intensities 
corresponding to m+1 pixels which are adjacent, and in the same row as, the 
interval of interest. Thus, for intervals on the lateral edges of region 80 
corresponding windows will extend laterally from the region of interest by as _ 
5 much as m/2 pixels. One such laterally extending window is identified by 

number 82. Pixels which are laterally positioned with respect to the region of 
interest 80 are similar to intervals within region 80 in that they have not yet 
been convoluted and interpolated. Therefore, convolution windows which 
laterally extend during the first pass of an image through hardware include 

10 only non-convoluted pixels. 

However, after intervals within a region of interest have been expanded 
in the x dimension, pixels in rows directly above and below the region of 
interest have not been expanded. Thus, upon the second pass of the image 
through the hardware to expand intervals in the y dimension, for intervals near 

15 the top and bottom of the region which has already been expanded in the x- 
dimension, suitable convolution windows cannot be selected. For example, 
for an interval in the top row of an x-dimension expanded region, when a 
convolution window including 5 pixels is chosen, while the pixels below the 
interval of interest and within the window have been expanded in the x- 

20 dimension like the interval of interest, the pixels above the interval of interest 
have not been similarly expanded in the x-dimension and therefore cannot be 
suitably used to convolute and interpolate the interval of interest. 

To overcome this problem, referring to Fig. 7, the region of interest 80 
is expanded to include m/2 rows of pixels above and m/2 rows of pixels below 

25 the initial region of interest resulting in expanded region 84. Thereafter, after 
expansion in the x dimension, expansion in the y dimension can be performed 
on intervals in rows corresponding to the original region of interest using 
convolution windows for top and bottom located pixels which extend into 
region 84 above region 80 and similarly into the region below region 80. 

30 Referring still to Figs. 6 and 10, in addition to receiving intensity signals 

/, for each pixel within the expanded region 84, at block 180 convolver 56 also 
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receives convolution filter coefficients a, from determiner 44 (see Fig. 5), filter 
size value m and factor n. Convolver 56 first selects pixel pairs adjacent each 
interval of interest and then selects a separate convolution window for each 
adjacent pixel on the expanded region of interest, and, at block 182, forms 
5 Equations 1 and 2 generating intermediate values and b 2 for the windows. 

At block 184 interpolator 58 receives values b, and b 2 and factor n and 
solves either Equation 3 or Equation 4 (depending on if factor n is odd or 
even) for each of k output pixels to determine pixel intensities 0, through O k 
Intensities 0 1 through O k are determined for each interval in the original 

10 region of interest 84 (see Fig. 7) and together intensities - O k define an 

interpolated image (i.e. the interpolated region expanded in the x dimension). 
The interpolated image is temporarily stored in buffer 60. Once the 
interpolated image has been completed, that image, including all pixel 
intensities, is provided to convolver 56 a second time. At block 186, for every 

1 5 interval between adjacent pixels in adjacent rows of the interpolated image 
which corresponded to rows in the initial region of interest 80 (see Fig. 7), 
convolver 56 identifies the adjacent pixels and selects a convolution column 
window for each adjacent pixel and convolves the intensities in the windows 
according to Equations 1 and 2 generating intermediate values b, and b 2 . At 

20 block 188, interpolator 58 receives values b 1 and b 2 , performs either Equation 
3 or Equation 4 (depending on if n is odd or even) and generates final output 
intensities which together define the final and magnified image. Computer 32 
(see Fig. 5) uses intensities O k to drive display 34 to display the magnified 
region of interest. 

25 Referring now to Fig. 9, the remaining hardware to facilitate the TDC 

method includes a 2D convolver 90 and a bi-linear interpolator 92. Referring 
also to Fig. 1 1 , at block 150 convolver 90 receives the image pixel intensities 
/, comprising the region of interest, the convolution filter coefficients a, , factor 
n and filter value m. Convolver 90 selects four adjacent pixels for each 

30 interpixel interval and then selects a separate convolution window for each 
interpixel interval within the region of interest. At block 152 convolver 90 
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performs Equations 18 through 21 to determine intermediate values b u , b 12 , 
.b 21 and b^. At block 152 interpolator 92 receives each of values b n through 
b^ and factor n for each interval in the region of interest and interpolates 
each set of four values b„ through b 22 to generate nxn pixel intensities O, by _ 
5 solving either Equation 22 or Equation 23, depending on if factor n is odd or 
even. Pixel intensities define the final output magnified image and can be 
used by computer 32 (see Fig. 5) to display the magnified image on display 
34. 

It should be understood that the methods and apparatuses described 
10 above are only exemplary and do not limit the scope of the invention, and that 
various modifications could be made by those skilled in the art that would fall 
under the scope of the invention. 

To apprise the public of the scope of this invention, we make the 
following claims : 
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What is claimed is: 

1 . A method to be used with a digital imaging system to facilitate 
magnification of a region of interest P1-P6 on an initial digital image wherein 
the region of interest P1-P6 includes a plurality of pixels of interest P r P 6 , 
adjacent portions of adjacent pixels of interest forming interpixel intervals P34 
of interest, each pixel of interest characterized by a pixel intensity l r l 6 , the 
system including convolution filter 56 and interpolation hardware 58, the 
method comprising the steps of: 

(A) for each interpixel interval of interest P34, identifying adjacent 
pixels P3, P4; 

(B) for each adjacent pixel: 

(i) selecting a convolution window 10 a , 10 b and, for each 
window: 

(a) determining 1 68 a convolution filter coefficient for 
each window pixel; 

(b) convoluting 182 the intensities of the pixels in the 
window as a function of the coefficients to generate an 
intermediate value b^ b 2 ; and 

(ii) interpolating 184 the intermediate values to generate at 
least two interpolated pixel intensities, the interpolated pixel 01- 
05 intensities together forming an interpolated image. 

2. The method of claim 1 wherein the initial pixel array is arranged 
in y columns and x rows and wherein the step of selecting includes the step 
of, for each interpixel interval P34 between pixels in adjacent columns, 
selecting a row convolution window 10a, 10b for each adjacent pixel 
consisting of pixels which are in the same row as the interpixel interval of 
interest P34 and wherein, after step (B)(ii) is completed, the method further 
includes the steps of: 
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for each interpixel interval between pixels in adjacent rows on the 
interpolated image 12, selecting a separate column convolution window for 
each adjacent pixel on the interpolated image, each column window including 
image pixels within the same column as, and proximate the interpixel interval _ 
5 of interest; 

for each column window: 

(i) determining 1 68 a convolution filter coefficient for each 
window pixel; 

(ii) convoluting 168 the pixel intensities in each window as a 
10 function of the coefficients to generate an intermediate value; 

and 

(Hi) interpolating 168 the intermediate values to generate a 
final pixel intensity, the final pixel intensities together forming the 
final image. 

15 3. The method of claim 2 wherein n is a magnification factor, the 

step of interpolating 184 to generate interpolated pixel intensities includes 
interpolating to generate n interpolated pixel intensities 01-05 and the step of 
interpolating 186 to generate final pixel intensities includes interpolating to 
generate n final intensities. 

20 

4. The method of claim 2 wherein the window 10a is an mx1 array 
and wherein an expanded region of interest B includes m/2 rows of pixels 
above and below the region of interest and the step of selecting row 
convolution windows for each interval of interest includes selecting row 

25 convolution windows for each interval in the expanded region of interest B. 

5. The method of claim 1 wherein each of the windows 1 0a, 1 0b 
includes the same number of pixels P1-P6. 
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6. The method of claim 1 wherein the image is to be magnified by 
a factor n and the step of determining convolution filter coefficients includes 
the steps of: 

calculating 162 linear interpolation coefficients; 
5 calculating 162 cubic interpolation coefficients; and 

mathematically combining 166, 168 the cubic interpolation coefficients 
and linear interpolation coefficients to determine the filter coefficients. 

7. The method of claim 6 wherein the step of mathematically 
combining 166, 168 includes the step of solving 168 the following equation: 

10 A=(A T A)- 1 A T C 

where A is a filter coefficient matrix, A is a linear interpolation matrix and C is 
a vector corresponding to the cubic interpolation coefficients. 

8. The method of claim 6 wherein the image is to be magnified by 

a factor n and the step of interpolating 184 includes the step of mathematically 
15 combining each of the intermediate values b lT b 2 as a function of factor n. 

9. The method of claim 8 wherein the step of mathematically 
combining includes the step of, where n is odd, solving the following equation: 

o k = — - — 2 J a i I i+——L a i 1 ^ 

n ,*=i * i-i 

and, where n is odd, solving the following equation: 
on ^ (/i-Jt + 0i)^ , (*-05)A , 

W ,-1 « i-l 
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10. The method of claim 1 wherein the convolution window 10a is 
one dimensional and includes m pixels and the step of convoiuting 182 
includes solving the following equations: 

/■I 

5 and 

m 

where b 1 and b 2 are the intermediate values, a, are the linear interpolation 
coefficients and / f are the initial pixel intensities l r l 6 of pixels in the convolution 
window 10a. 

10 11. The method of claim 1 wherein the hardware 90, 92 is capable 

of performing high speed two dimensional convolution and bilinear 
interpolation, the step of selecting convolution windows including the step of 
selecting four mxm windows (one illustrated as P|,-P mm ), each window ncluding 
the interval of interest. 

15 12. The method of claim 1 1 wherein the step of convoiuting 152 

includes solving the following equations: 

m m 
r-l >»! 

m m 
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1 3. The method of claim 1 2 wherein the step of interpolating 1 54 
includes the steps of, where n is odd, solving the following equation: 



_ („-/+ lX*-y+i) Qi-i + i)C/-i) L 

°9 ~ ~i 6 U T ~2 6 12 + 

2 &21 + 2 ^22 

n 



and where n is even, solving the following equation: 

_ (w-f + 0i)(it-y + 0i) Qi-i + ftS)(/-0.5) L 
°(f " *n + -J b n + 

(/-0.5)(tt-y + 0.5) L ((- 0.5)0 ^Qj) . 



14. A pulse sequencing program to be used with a digital imaging 
system to facilitate magnification of a region of interest on an initial digital 
image wherein the region of interest includes a plurality of pixels of interest, 
adjacent portions of adjacent pixels of interest forming interpixel intervals of 
interest, each pixel of interest characterized by a pixel intensity, the 
sequencing program performing the steps of: 

(A) for each interpixel interval of interest, identifying adjacent pixels; 

(B) for each adjacent pixel: 

(i) selecting a convolution window and for each window: 

(a) determining a convolution filter coefficient for each 
window pixel; 

(b) convoluting the intensities of the pixels in the 
window using the coefficients to generate an intermediate 
value; and 
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(ii) interpolating the intermediate values to generate at least 
two interpolated pixel intensities, the interpolated pixel intensities 
together forming an interpolated image. 

15. The program of claim 14 wherein the initial pixel array is 
arranged in y columns and x rows and wherein the step of selecting includes 
the step of, for each interval of interest between pixels in adjacent columns, 
selecting a row convolution window for each adjacent pixel consisting of pixels 
which are in the same row as the interval of interest and wherein, after step 
(B)(ii) is completed, the program further includes the steps of: 

for each interval between pixels in adjacent rows on the interpolated 
image, selecting a separate column convolution window for each adjacent 
pixel on the interpolated image, each column window including image pixels 
within the same column as, and proximate the interval; 

for each column window: 

(i) determining a convolution filter coefficient for each 
window pixel; 

(ii) convoluting the pixel intensities in each window using the 
coefficients to generate an intermediate value; and 

(iii) interpolating the intermediate values to generate a final 
pixel intensity, the final pixel intensities together forming the final 
image. 

1 6. The program of claim 1 5 wherein n is a magnification factor, the 
step of interpolating to generate interpolated pixel intensities includes 
interpolating to generate n interpolated pixel intensities and the step of 
interpolating to generate final pixel intensities includes interpolating to 
generate n final intensities. 

17. The program of claim 15 wherein each window is an mx1 array 
and wherein an expanded region of interest includes m/2 rows of pixels above 
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and below the region of interest and the step of selecting first and second 
convolution windows for each interval of interest includes selecting first and 
second row convolution windows for each interval in the expanded region of 
interest. 

18. The program of claim 14 wherein each of windows includes the 
same number of pixels. 

19. The program of claim 14 wherein the image is to be magnified 
by a factor n and the step of determining convolution filter coefficients 
includes the steps of: 

calculating linear interpolation coefficients; 
calculating cubic interpolation coefficients; and 
mathematically combining the cubic interpolation coefficients and linear 
interpolation coefficients to determine the filter coefficients. 

20. The program of claim 19 wherein the step of mathematically 
combining includes the step of solving the following equation: 

A=(A T A)' 1 A T C 

where A is a filter coefficient matrix, A is a linear interpolation matrix and C is 
a vector corresponding to the cubic interpolation coefficients. 

21. The program of claim 19 wherein the image is to be magnified 
by a factor n and the step of interpolating includes the step of mathematically 
combining each of the intermediate values as a function of factor n. 

22. The program of claim 21 wherein the step of mathematically 
combining includes the step, where n is odd, of solving the following equation: 
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0* - — - — z- 0 - 7 - + — — 2- a ' 7 .» 
« .=1 « ,=! 

and where n is even, solving the following equation: 

o k = - — - — -2>,/, -^Z 

23. The program of claim 14 wherein the convolution window is one 
dimensional and includes m pixels and the step of convoluting includes 
solving the following equations: 

m 

1=1 

m 

where b1 and b2 are the intermediate values, ai are the linear interpolation 
coefficients and /, are the initial pixel intensities of pixels in the in the 
convolution window. 

24. The program of claim 1 4 wherein the hardware is capable of 
performing high speed two dimensional convolution and bilinear interpolation, 
the step of selecting convolution windows includes the step of selecting four 
mxm window including the pixel of interest. 

25. The program of claim 23 wherein the step of convoluting 
includes solving the following equations: 

m m 

M jm\ 
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6 12 =IZV/M 



m m 



5 26. An apparatus to be used with a digital imaging system to 

facilitate magnification by a factor n of a region of interest on an initial digital 
image wherein the region of interest includes a plurality of pixels of interest, 
adjacent portions of adjacent pixels of interest forming interpixel intervals of 
interest, each pixel of interest characterized by a pixel intensity, the apparatus 
10 comprising: 

a first calculator which, based on factor n, generates cubic coefficients; 
a second calculator which, based on factor n, determines interpolation 
coefficients; 

a determiner which mathematically combines the cubic and 
15 interpolation coefficients to determine a plurality of convolution filter 
coefficients; 

a convolver which, for each interval of interest: 

identifies pixels adjacent interpixel intervals of interest; 
for each adjacent pixel, selects a convolution window; 
20 for each convolution window, convolutes the pixel 

intensities of the pixels in each window using the convolution 
coefficients to generate an intermediate value; and 
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an interpolator which interpolates the intermediate values to generate 
. at least two interpolated pixel intensities, the interpolated pixel intensities 
together forming an interpolated image. 

27. The apparatus of claim 26 wherein the initial pixel array is 

5 arranged in y columns and x rows and wherein when selecting, the convolver, 
for each interval of interest between pixels in adjacent columns, selects a row 
convolution window for each adjacent pixel consisting of pixels which are in 
the same row as the interval of interest and wherein, after the interpolated 
image has been generated, the interpolated image is provided to the 

10 convolver which, for each interval between pixels in adjacent rows on the 
interpolated image, selects a separate column convolution window for each 
adjacent pixel on the interpolated image, each column window including 
adjacent image pixels within the same column as, and proximate the interval, 
for each column window, convolutes the pixel intensities in each window using 

15 the coefficients to generate an intermediate value, the intermediate values 
provided to the interpolator which again interpolates the intermediate values 
to generate a final pixel intensity, the final pixel intensities together forming 
the final image. 

28. The apparatus of claim 27 wherein each window includes an 
20 mx1 array and wherein an expanded region of interest includes m/2 rows of 

pixels above and below the region of interest and the convolver selects first 
and second row convolution windows for each interval in the expanded region 
of interest. 

29. The apparatus of claim 26 wherein each of the first and second 
25 sub-windows include m pixels. 

30. The apparatus of claim 26 wherein the determiner 
mathematically combines by solving the following equation: 
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A=(A T A)' 1 A T C 

where A is a filter coefficient matrix, A is a linear interpolation matrix and C is 
a vector corresponding to the cubic interpolation coefficients. 

31 . The apparatus of claim 30 wherein the convolver convolves by 
5 solving the following equations: 

m 

1=1 

and 

m 

where b1 and b2 are the intermediate values, a, are the linear interpolation 
10 coefficients and I t are the initial pixel intensities of pixels in the convolution 
window. 

32. The apparatus of claim 31 wherein the interpolator interpolates 
by, where n is odd, solving the following equation: 

°k = — - — L a i J < + — — IVw 

15 and where n is even, solving the following equation: 
O k 2-^1+ — - — 

33. The apparatus of claim 26 wherein the convolver selects four 
two-dimensional convolution windows having dimensions mxm, each window 
including the interval of interest. 

20 34. The apparatus of claim 33 wherein the convolver convolves by 

solving the following equations: 
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m m 



til m 



m m 
/=1 y=l 



m m 

/=i y-i 



5 35. The apparatus of claim 34 wherein the interpolator interpolates 

by, where n is odd, solving the following equation: 

(«-/+!)(«-;+ 1) , , ( w -i+i)(/-i) L 
C/„ -j o„ + j 6, 2 + 



n 

12 *2! + ~2 °22 

n n 



and, where n is even, solving: 



_ (n-i + 05)(n-j + 0S) L (n-i + 05)(J-05) u 
O v ~T" -o„ +- — 0,2 + 

(/-0i)(H-y+0i) L , (£-05)0-05) , 

~2 f> 2 , + -j »22 
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